Skip to content

feat: add graph runtime api#9

Draft
gongchensu wants to merge 5 commits into
masterfrom
feat/graph-runtime-api
Draft

feat: add graph runtime api#9
gongchensu wants to merge 5 commits into
masterfrom
feat/graph-runtime-api

Conversation

@gongchensu

@gongchensu gongchensu commented Jun 24, 2026 •
edited by voltjia
Loading

Copy link
Copy Markdown

Summary

  • Adds generated C++ runtime declarations and dispatch support for stream capture and graph lifecycle APIs in scripts/generate_public_headers.py.
  • Adds native graph API coverage for CUDA-like backends in src/native/cuda/nvidia/runtime_.h, src/native/cuda/iluvatar/runtime_.h, src/native/cuda/metax/runtime_.h, and src/native/cuda/moore/runtime_.h.
  • Adds an adaptive graph runtime test in tests/test_native_graph.cc, wired through tests/CMakeLists.txt.

Motivation

InfiniRT should expose CUDA Runtime API-aligned graph capture and graph launch APIs through the generated C++ runtime API, without introducing a C API. This is needed by the standalone InfiniRT graph bridge work in InfiniCore.

Related: InfiniTensor/InfiniCore#1351

Type of Change

  • feat - new feature / new operator / new platform
  • fix - bug fix
  • perf - performance improvement (no behavioral change)
  • refactor - code restructuring without behavior change
  • test - adding or fixing tests only
  • docs - documentation only
  • build / ci - build system or CI configuration
  • chore - tooling, formatting, or other non-code changes
  • Breaking change (requires a ! in the Conventional Commits prefix or a BREAKING CHANGE: footer)

Platforms Affected

  • CPU (WITH_CPU)
  • NVIDIA (WITH_NVIDIA)
  • Iluvatar (WITH_ILUVATAR)
  • MetaX (WITH_METAX)
  • Cambricon (WITH_CAMBRICON)
  • Moore (WITH_MOORE)
  • Ascend (WITH_ASCEND)
  • PyTorch C++ bindings (WITH_TORCH)
  • Build system / CMake / CI
  • Python bindings / user-facing API

Smoke Test Result

# ssh nvidia, image: infinirt-verify:nvidia-latest
clang-format --dry-run --Werror include/infini/rt.h tests/test_native_graph.cc
cmake -B build \
  -DWITH_CPU=ON \
  -DWITH_NVIDIA=ON \
  -DINFINI_RT_BUILD_TESTING=ON \
  -DCMAKE_BUILD_TYPE=Release \
  -DCMAKE_INSTALL_PREFIX=/workspace/infini-rt-prefix
cmake --build build -j$(nproc)
ctest --test-dir build --output-on-failure

100% tests passed, 0 tests failed out of 8
Passed tests include test_nvidia_graph and test_runtime_dispatch.

Test Results on Supported Platforms

Platform Affected Build / Smoke Result Full Result / Notes
NVIDIA Yes smoke passed ctest passed: 8/8
Iluvatar Yes not run No Iluvatar validation in this round
MetaX Yes not run No MetaX validation in this round; API currently returns unsupported for graph capture
Cambricon No N/A - not affected N/A
Moore Yes not run No Moore validation in this round; API currently returns unsupported for graph capture
Ascend No N/A - not affected N/A
Full `pytest` output (optional) 
N/A

Benchmark / Performance Impact

N/A. This PR adds runtime API surface and tests; it does not tune kernel performance.

Notes for Reviewers

  • This PR intentionally keeps InfiniRT as C++ API only. No C API is added.
  • The public runtime graph API is generated under infini::rt::runtime.
  • MetaX and Moore expose the graph entry points for API completeness but currently return unsupported until native graph support is available.

@gongchensu gongchensu self-assigned this Jun 24, 2026
@gongchensu gongchensu requested a review from voltjia June 24, 2026 04:24
voltjia
voltjia requested changes Jun 24, 2026
View reviewed changes
Comment thread tests/CMakeLists.txt Outdated
Comment thread tests/test_nvidia_graph.cc Outdated
Comment thread src/graph.h Outdated
Comment thread src/native/cuda/runtime_.h Outdated
Comment thread src/native/cuda/runtime_.h Outdated
Comment thread src/native/cpu/runtime_.h Outdated
Comment thread src/native/ascend/runtime_.h Outdated
Comment thread src/native/ascend/runtime_.h Outdated
Comment thread src/native/cambricon/runtime_.h Outdated
Comment thread src/native/cambricon/runtime_.h Outdated
@voltjia voltjia requested a review from Ziminli June 24, 2026 08:49
Ziminli
Ziminli requested changes Jun 24, 2026
View reviewed changes
Comment thread src/native/cuda/runtime_.h
@gongchensu gongchensu force-pushed the feat/graph-runtime-api branch 3 times, most recently from aae1676 to a4bac7f Compare June 25, 2026 09:13
voltjia
voltjia requested changes Jun 26, 2026
View reviewed changes
Comment thread src/graph.h Outdated
Comment on lines +32 to +33
Device::Type device_type_{Device::Type::kCpu};
void* raw_{nullptr};

@voltjia voltjia Jun 26, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

之间空一行吧。

Comment thread src/graph.h Outdated
Comment on lines +50 to +51
Device::Type device_type_{Device::Type::kCpu};
void* raw_{nullptr};

@voltjia voltjia Jun 26, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

之间空一行吧。

Comment thread src/graph.h Outdated
Comment on lines +68 to +69
Device::Type device_type_{Device::Type::kCpu};
void* raw_{nullptr};

@voltjia voltjia Jun 26, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

之间空一行吧。

@gongchensu gongchensu force-pushed the feat/graph-runtime-api branch from a4bac7f to b9c8bad Compare June 30, 2026 06:31
This was referenced Jul 3, 2026
Merged
Merged
gongchensu and others added 3 commits July 3, 2026 16:07
@gongchensu @voltjia
feat: add graph runtime api
a10f952
@gongchensu @voltjia
feat: add graph c api
2cb2cd2
@voltjia @spike-zhu
fix: align graph runtime API with runtime namespace
81823a4 
* feat!: align runtime API and add runtime dispatch (#11)

* Align runtime API with generated wrappers

* Add default runtime dispatch specialization

* Refactor runtime dispatch namespace

* Use Abseil status for runtime device API

* Revert "Use Abseil status for runtime device API"

This reverts commit a26ddff.

* Address runtime dispatch review feedback

* Keep runtime API list in generator

* Add TensorView constructor guard test

* Align runtime memcpy kind constants with CUDA API

* Use CUDA-style runtime memcpy constants

* Use CUDA-style runtime memcpy constants

* Move TensorView tests back into core test

* Remove standalone TensorView test target

* Remove standalone TensorView test file

* Use fully qualified runtime API names in README

* style: format runtime dispatch test

* feat: refactor InfiniCore CPU runtime to InfiniRT (#8)

Co-authored-by: Jiacheng Huang <huangjiacheng0709@outlook.com>

* feat: add platform-adaptive runtime tests (#15)

* feat: add runtime backend API foundation (#14)

---------

Co-authored-by: spike-zhu <74974704+spike-zhu@users.noreply.github.com>
@voltjia voltjia force-pushed the feat/graph-runtime-api branch 2 times, most recently from 81df7b1 to 9f82a06 Compare July 3, 2026 08:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@voltjia voltjia voltjia requested changes
@Ziminli Ziminli Ziminli requested changes

Requested changes must be addressed to merge this pull request.

Assignees

@gongchensu gongchensu

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@gongchensu @voltjia @Ziminli